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Introduction 


• Realities of Man Rated systems 

• Realities of centralized processing 

• Criticality independent improvements 

• Criticality dependent improvements 

• Criticality dependent architecture decisions 

• Partitioning by criticality 

• Mission critical development option 
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High Criticality: Necessary Caution 


• What high criticality means for Space Shuttle 

- Human life is dependent on correct operation 

• Drives emphasis on quality, reliability, safety 

- Controlled, predictable, and repeatable development processes 

- Analysis of all software errors for flight safety impact 

- Methods open to defect cause analysis 

- Development and test tools also treated as critical 

- Flight Support requires near immediate response 

• Corrections or work arounds expected during operational use 

• Delivery in hours 

• Developed following a stringent process 
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Software Test Environment Complexity 


Centralized Processing 
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Less Critical: Caution & Culture 


• Flexibility should be permitted when the consequences of software failure are 
non-life threatening 

- Expected software quality is consequence driven 

• Less costly development methods 

• Less costly defect control process 

• Less oversight of development processes 

- Flight Support levels are consequence driven 

• Less extraordinary support requirements 

• Recovery more important than immediate understanding of cause 

• Corrections by release, not by patch 

• Flight Critical culture may require actions which are inconsistent with failure 
consequences 

- Decades of centralized processing have institutionalized high criticality 
thinking 
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Choosing COTS Software Trade Candidates 


Trade for High Criticality Usage 

Software Production Process 
Flight Support 
Technical Suitability 
System Compatibility 
Product Longevity Assessment 
Technical Support 
Cost 


Trade for Lower Criticality Usage 

Flight Support 
Technical Suitability 
System Compatibility 
Product Longevity Assessment 
Technical Support 
Cost 


Equivalent criteria 
Unique criteria 


Scalable criteria 


USA 


10/27/99 - ATWG.ppt 


Page 6 


United Space Alliance 


COTS Decision Guidance 


Selecting COTS/MOTS for high criticality functions should require greater technical 
insight and stronger risk management planning than for lower criticality functions 

High Criticality 

Is Certification Plan adequate for criticality? 

Will vendor disclose defects found by other users? 

Are there adequate measures of quality and reliability? 

Will vendor disclose development methods? 

Is vendor willing to escrow source code? 

Visibility into design and code? 

Is Flight Support Plan adequate? 

Is technical support plan adequate? 

Is the risk management plan for loss of support adequate? 


Lower Criticality 

Is Certification Plan adequate for criticality? 

Will vendor disclose defects found by other users? 

Is vendor willing to escrow source code? 

Is Flight Support Plan adequate? 

Is technical support plan adequate? 

Is the risk management plan for loss of support adequate? 
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Realities of Centralized Processing 


Flight Software 
Changes 


Mission 

Reconfiguration 

\T 


GPC 

Softwa re 


Flight 

Reconfiguration 



Flight Critical 
Development Processes 

] Requirements 
I I Design 

1 I Code 

1. I Verification & System Test 

I □ Recon Test 
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Criticality Independent Improvements 


• Technology modernization enables process improvements independent of 
software criticality 

- Requirements Definition and Analysis Phase 

• On-line requirements and on-line reviews 

• Requirements prototyping 

- Software Development and Verification Phase 

• Visual presentation of design 

• Design directly coupled to code 

• Modern desk-top development tools 

• Automatic path/segment test tools including coverage analysis 

• LAN based simulations 

• Automatic test report generation 

• Potential development cost reduction of ~10% for new avionics software 
compared to current methods 
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Criticality Dependent Changes 


• Candidate process changes based on a criticality partitioned system 

- V&V testing to vary with criticality of functionality 

- Redundant testing coverage (like today) for Criticality 1 software 

- Less than full shall & path coverage for lower criticality software 

• Reduced testing documentation 

• Random sampling of V&V test results for NASA review 

- Test philosophies to be evaluated with various combinations of V&V testing 
and analysis or audit 


• Potential development cost reduction of -22% for new avionics software 
compared to current methods 
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Criticality Isolation Is Difficult to Achieve 


High Criticality 


General 

Purpose 

Computers 

GN&C 

Non-GN&C 


Displays Payloads 
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Partitioning by Criticality 


Flight Critical 


Mission Critical 

GPC’s 


Processor 


Flight Critical 
Development Process 

1. — 1 Requirements 

I 1 Design 


Code 


J Detailed Verification 


System Test 


Development Process 

□ Requirements 
□ Design 

1 1 Code 

1... . 1 Detailed Verification 
□ System T est 


Recon Test 


□ Recon Test 
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Selecting a Process to Match Criticality 


• Sample predicted defects remaining at first flight (per 100K SLOCS) 

• SAMPLE DATA - Not final quantitative values 


Errors Remaining 
Inserted Errors/KSLOC 10 

Removed in Inspection 65% of 

Development Test 55% of 

Software Integration Test 50% of 

After V&V Testing 

Criticality 1 Test Philosophy 
Criticality 2 Test Philosophy 
Criticality 3 Test Philosophy 
After Integrated Avionics Verification 
Criticality 1 Test Philosophy 


Total 

Remaining 

Remaining 


1,000 Errors 
350 Errors 
150 Errors 
75 Errors 


80% of Remaining 
60% of Remaining 
40% of Remaining 
Testing 

65% of Remaining 


15 Errors 
30 Errors 
45 Errors 

5 Errors 
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Life Cycle Support 


GPC S/W 

System Software $ 

Flight Critical $ 

- Non Flight Critical $ 

S/W Prod. Facility $ 

$$$ 


New Avionics S/W 

COTS RTOS $ 

+ Non-Flight Critical $ 
S/W Dev. Facility $ 

$$ 


Partitioning for criticality limits support costs for new avionics 
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Summary 


• Man Rated systems require added caution 

• Distributed processing increases the software verification boundaries 

• Select COTS with care and take appropriate risk mitigation actions 

• Single criticality forces a single process 

• Partitioning enables flexibility in process selection 

• Appropriate process tailoring necessary to yield required costs and quality 

• Criticality partitioning is key to controlling costs 
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